Designing various component analysis at will 
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Abstract 

This paper provides a generic framework of compo- 
nent analysis ( CA ) methods introducing a new expres- 
sion for scatter matrices and Gram matrices, called 
Generalized Pairwise Expression ( GPE). This expres- 
sion is quite compact but highly powerful: The frame- 
work includes not only (1) the standard CA methods but 
also (2) several regularization techniques, (3) weighted 
extensions, (4) some clustering methods, and (5) their 
semi-supervised extensions. This paper also presents 
quite a simple methodology for designing a desired 
CA method from the proposed framework: Adopting 
the known GPEs as templates, and generating a new 
method by combining these templates appropriately. 


1 Introduction 

Component analysis (CA) is traditional, quite sim- 
ple but might be one of the powerful tools to obtain a 
hidden structure embedded in the data. Recent reports 
showed its effectiveness for several tasks in computer 
vision and pattern recognition. Principal component 
analysis (PCA), Fisher discriminant analysis (FDA), 
multiple linear regression (MLR), and canonical cor- 
relation analysis (CCA) are well known as standard 
CA methods 0]. They can be formulated as a gen- 
eralized eigenvalue problem of a scatter matrix or an 
augmented matrix composed of several scatter matri- 
ces (2] [3] . Kernel C A methods as kernelized extensions 
of those standard methods have been also developed to 
deal with non-vector samples and non-linear analysis, 
which can be formulated as a generalized eigenvalue 
problem of Gram matrices, instead of scatter matrices 
112] [3). Kernel CA often needs some regularization tech- 
niques such as ^2-norm regularization to inhibit overra- 
ting and Laplacian regularization to fit underlying data 
manifolds smoothly. In addition, improvements of ro- 
bustness against outliers and separately distributed sam- 
ples and their extensions to semi-supervised analysis 


have been considered. 

Although a lot of CA methods and several trials 
to unify these methods have been presented so far 
E][3]|4]|5], freely designing a tailor-made method of CA 
for a specific purpose or domain still remains an open 
problem. Until now, researchers have had to choose one 
of the existing methods that seems best to address the 
problem of interest, or had to laboriously develop a new 
analysis method tailored specifically for that purpose. 

In view of the above discussions, this paper provides 
a new expression of scatter matrices and Gram matrices, 
which we call generalized pairwise expression ( GPE) to 
make it easy to design a new CA method with desired 
properties. The methodology is quite simple: adopt- 
ing the above mentioned special cases as templates, 
and generating a new method by combining these 
templates appropriately. This characteristics has not 
been discussed yet in any previous researches to our 
best knowledge. It is also possible to individually se- 
lect and arrange samples for calculating the scatter ma- 
trices of the methods to be combined, which enables 
us to extend CA methods to semi-supervised ones and 
multi-modal ones. 
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